Decision-Dependent Distributionally Robust Markov Decision Process Method in Dynamic Epidemic Control

In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. While Markov Decision Processes (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spreading according to the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost. [ FROM AUTHOR] Copyright of IISE Transactions is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)

Marginal Cost of Energy Volatility in the Brazilian Interconnected Power System

From 2018 to 2022, on average, 70% of the Brazilian effective electric generation was produced by hydropower, 10% by wind power, and 20% by thermal power plants. Over the last five years, Brazil suffered from a series of severe droughts. As a result, hydropower generation was reduced, but demand growth was also declined as results of the COVID-19 pandemic and economic recession. From 2012 to 2022, the Brazilian reservoir system operated with, on average, only 40% of the active storage, but storage recovered to normal levels in the first three months of 2022. Despite large capacity of storage reservoirs, high volatility of the marginal cost of energy was observed in recent years. In this paper, we used two optimization models, NEWAVE and HIDROTERM for our study. These two models were previously developed for mid-range planning of the operation of the Brazilian interconnected power system. We used these two models to optimize the operation and compared the results with observed operational records for the period of 2018-2022. NEWAVE is a stochastic dual dynamic programming model which aggregates the system into four subsystems and 12 equivalent reservoirs. HIDROTERM is a nonlinear programming model that considers each of the 167 individual hydropower plants of the system. The main purposes of the comparison are to assess cooperation opportunities with the use of both models and better understand the impacts of increasing uncertainties, seasonality of inflows and winds, demand forecasts, decisions about storage in reservoirs, and thermal production on energy prices. © World Environmental and Water Resources Congress 2023.All rights reserved

2022 the 4th International Conference on Modeling, Simulation, Optimization and Algorithm[ICMSOA 2022], 12 November, 2022

ABOUT ICMSOA2022Organized by Yaseen Academy, 2022 The 2nd International Conference on Modeling, Simulation, Optimization and Algorithm (ICMSOA 2022), which was planned to be held during 11-13 November, 2022 at Sanya, Hainan Province, China. Due to the travel restrictions caused by covid, the participants joined the conference online via Tencent Meeting at 12 November, 2022. The Conference looks for significant contributions to related fields of Modeling, Simulation, Optimization and Algorithm. The aim of the conference is to provide a platform to the researchers and practitioners from both academia as well as industry to meet and share cutting-edge development in the field.CALL FOR PAPERSPlease make sure your submission is in related areas of the following general topics. The topics include, but are not limited to:Simulation modeling theory and technology, Computational modeling and simulation, System modeling and simulation, Device/VLSI modeling and simulation, Control theory and applications, Military Technology Simulation, Aerospace technology simulation, Information engineering simulation, Energy Engineering Simulation, Manufacturing Simulation, Intelligent engineering simulation, Building engineering simulation, Electromagnetic field simulation, Material engineering simulation, Visual simulation, Fluid mechanics engineering simulation, Manufacturing simulation technology, Simulation architecture, Simulation software platform and Intelligent Optimization Algorithm, Dynamic Programming, Ant Colony Optimization, Genetic Algorithm, Simulated Annealing Algorithm, Tabu Search Algorithm, Ant Colony System Algorithm, Hybrid Optimization Algorithm in other related areas.The conference was begun at 10:00am, ended at 17:30am, 12 November, 2022. There were 77 participants in total, 2 keynote speakers and 17 invited oral speakers, Assoc. Prof. Jinyang Xu from Shanghai Jiaotong Univeristy in China and Dr. Victor Koledov from Innowledgement GmbH in Germany delivered their keynote speeches, each speech cost about 50 minutes, including the questions&discussion time.On behalf of the conference organizing committee, we'd like to acknowledge the unstinting support from our colleagues at Yaseen Academy, all Technical Program Members, speakers, reviewers, and all the participants for their sincere support.Conference Organizing CommitteeICMSOA 2022List of Conference General Chair, Program Chair, Conference Committee Chair Members, International Technical Committee Members, International Reviewers are available in this Pdf.

Hierarchical Multi-agent Model for Reinforced Medical Resource Allocation with Imperfect Information

With the advent of the COVID-19 pandemic, the shortage in medical resources became increasingly more evident. Therefore, efficient strategies for medical resource allocation are urgently needed. However, conventional rule-based methods employed by public health experts have limited capability in dealing with the complex and dynamic pandemic-spreading situation. In addition, model-based optimization methods such as dynamic programming (DP) fail to work since we cannot obtain a precise model in real-world situations most of the time. Model-free reinforcement learning (RL) is a powerful tool for decision-making;however, three key challenges exist in solving this problem via RL: (1) complex situations and countless choices for decision-making in the real world;(2) imperfect information due to the latency of pandemic spreading;and (3) limitations on conducting experiments in the real world since we cannot set up pandemic outbreaks arbitrarily. In this article, we propose a hierarchical RL framework with several specially designed components. We design a decomposed action space with a corresponding training algorithm to deal with the countless choices, ensuring efficient and real-time strategies. We design a recurrent neural network-based framework to utilize the imperfect information obtained from the environment. We also design a multi-agent voting method, which modifies the decision-making process considering the randomness during model training and, thus, improves the performance. We build a pandemic-spreading simulator based on real-world data, serving as the experimental platform. We then conduct extensive experiments. The results show that our method outperforms all baselines, which reduces infections and deaths by 14.25% on average without the multi-agent voting method and up to 15.44% with it. © 2022 Association for Computing Machinery.

Inferring Phylogenetic Relationships using the Smith-Waterman Algorithm and Hierarchical Clustering

All biological species undergo change over time due to the evolutionary process. These changes can occur rapidly and unpredictably. Due to their high potential to spread quickly, it is critical to be able to monitor changes and detect viral variants. Phylogenetic trees serve as good methods to study evolutionary relationships. Complex big data in biomedicine is plentiful in regards to viral data. In this paper, we analyze phylogenetic trees with reference to viruses and conduct dynamic programming using the Smith-Waterman algorithm, followed by hierarchical clustering. This methodology constitutes an intelligent approach for data mining, paving the way for examining variations in SARS-Cov-2, which in turn can help to discover knowledge potentially useful in biomedicine. © 2022 IEEE.

Pandemic portfolio choice

COVID-19 has taught us that a pandemic can significantly increase biometric risk and at the same time trigger crashes of the stock market. Taking these potential co-movements of financial and non-financial risks into account, we study the portfolio problem of an agent who is aware that a future pandemic can affect her health and personal finances. The corresponding stochastic dynamic optimization problem is complex: It is characterized by a system of Hamilton-Jacobi-Bellman equations which are coupled with optimality conditions that are only given implicitly. We prove that the agent's value function and optimal policies are determined by the unique global solution to a system of non-linear ordinary differential equations. We show that the optimal portfolio strategy is significantly affected by the mere threat of a potential pandemic.

Dynamics of COVID-19 Using SEIQR Epidemic Model

The major goal of this study is to create an optimal technique for managing COVID-19 spread by transforming the SEIQR model into a dynamic (multistage) programming problem with continuous and discrete time-varying transmission rates as optimizing variables. We have developed an optimal control problem for a discrete-time, deterministic susceptible class (S), exposed class (E), infected class (I), quarantined class (Q), and recovered class (R) epidemic with a finite time horizon. The problem involves finding the minimum objective function of a controlled process subject to the constraints of limited resources. For our model, we present a new technique based on dynamic programming problem solutions that can be used to minimize infection rate and maximize recovery rate. We developed suitable conditions for obtaining monotonic solutions and proposed a dynamic programming model to obtain optimal transmission rate sequences. We explored the positivity and unique solvability nature of these implicit and explicit time-discrete models. According to our findings, isolating the affected humans can limit the danger of COVID-19 spreading in the future.

Optimal selection of COVID-19 vaccination sites in the Philippines at the municipal level.

In this work, we present an approach to determine the optimal location of coronavirus disease 2019 (COVID-19) vaccination sites at the municipal level. We assume that each municipality is subdivided into smaller administrative units, which we refer to as barangays. The proposed method solves a minimization problem arising from a facility location problem, which is formulated based on the proximity of the vaccination sites to the barangays, the number of COVID-19 cases, and the population densities of the barangays. These objectives are formulated as a single optimization problem. As an alternative decision support tool, we develop a bi-objective optimization problem that considers distance and population coverage. Lastly, we propose a dynamic optimization approach that recalculates the optimal vaccination sites to account for the changes in the population of the barangays that have completed their vaccination program. A numerical scheme that solves the optimization problems is presented and the detailed description of the algorithms, which are coded in Python and MATLAB, are uploaded to a public repository. As an illustration, we apply our method to determine the optimal location of vaccination sites in San Juan, a municipality in the province of Batangas, in the Philippines. We hope that this study may guide the local government units in coming up with strategic and accessible plans for vaccine administration.

Edge Computing and Lightning Network Empowered Secure Food Supply Management

The recent COVID-19 pandemic has highlighted the importance of food safety and supply chain governance. In other words, we need to ensure traceability along the supply chain and support high-frequency transactions, effective data collections, etc. Thus, we posit the potential of using a lightning network, which is a decentralized traceable paradigm for achieving high-frequency transactions in blockchain-based systems. In addition, we also utilize edge computing to help facilitate data collection. However, a key challenge in securing food supplies is determining the optimal global transaction path in the lightning network while achieving efficiency and meeting the dynamic nature of food supply management. Thus, we propose a blockchain-edge scheme that utilizes our proposed dynamic programming to produce optimal solutions for selecting global transaction paths. Specifically, our scheme optimizes routing fees under existing constraints (e.g., transmission cost, computing resource consumption, and lightning network balance). The findings from our evaluations demonstrate the utility of our proposed approach in facilitating food safety management.

ARPIP: Ancestral sequence Reconstruction with insertions and deletions under the Poisson Indel Process.

Modern phylogenetic methods allow inference of ancestral molecular sequences given an alignment and phylogeny relating present day sequences. This provides insight into the evolutionary history of molecules, helping to understand gene function and to study biological processes such as adaptation and convergent evolution across a variety of applications. Here we propose a dynamic programming algorithm for fast joint likelihood-based reconstruction of ancestral sequences under the Poisson Indel Process (PIP). Unlike previous approaches, our method, named ARPIP, enables the reconstruction with insertions and deletions based on an explicit indel model. Consequently, inferred indel events have an explicit biological interpretation. Likelihood computation is achieved in linear time with respect to the number of sequences. Our method consists of two steps, namely finding the most probable indel points and reconstructing ancestral sequences. First, we find the most likely indel points and prune the phylogeny to reflect the insertion and deletion events per site. Second, we infer the ancestral states on the pruned subtree in a manner similar to FastML. We applied ARPIP on simulated datasets and on real data from the Betacoronavirus genus. ARPIP reconstructs both the indel events and substitutions with a high degree of accuracy. Our method fares well when compared to established state-of-the-art methods such as FastML and PAML. Moreover, the method can be extended to explore both optimal and suboptimal reconstructions, include rate heterogeneity through time and more. We believe it will expand the range of novel applications of ancestral sequence reconstruction.

Control of COVID-19 transmission dynamics, a game theoretical approach

We analyze a mathematical model of COVID-19 transmission control, which includes the interactions among different groups of the population: vaccinated, susceptible, exposed, infectious, super-spreaders, hospitalized and fatality, based on a system of ordinary differential equations, which describes compartment model of a disease and its treatment. The aim of the model is to predict the development disease under different types of treatment during some fixed time period. We develop a game theoretic approach and a dual dynamic programming method to formulate optimal conditions of the treatment for an administration of a vaccine. Next, we calculate numerically an optimal treatment. © 2022, The Author(s), under exclusive licence to Springer Nature B.V.

Novel Artificial Intelligence-Dynamic Programming on Infrared Thermometer Based on Internet of Things (IoT)

According to the COVID-19 epidemic, the world has completely changed to new norm life. However, until 2022, people are still facing COVID-19 and its spreading and fast infection to the human body. Healthcare workers are on the front lines and are at higher risk of contracting COVID-19 than other occupations because they must be in close contact with the patient who risks virus diseases. The paper proposes the novel artificial intelligence (AI)-dynamic programming algorithm on infrared Thermometer based on the Internet of things (IoT) to support the medical personnel. The proposed novel thermometer is divided into three main sections, which are 1) Temperature sensing device, 2) Embedded dynamic programming algorithm, and 3) IoT communication platform. The innovation was designed using dynamic programming algorithm embedment, reducing complex and repetitive processing errors and fast computation. Moreover, it was tested according to the research methodology way. The temperatures were collected within the controlled condition test of time, environment condition, and same body organ of volunteer according to the various distances. The experimental results came out with three classified zones: best, moderate, and ineffective spaces. In addition, the discussions were also included about the complication factors about sensor's accuracy detection, such as angle detection, target distance, and focusing of wireless infrared Thermometer. © 2022 Wydawnictwo SIGMA-NOT. All rights reserved.

Foresighted medical resources allocation during an epidemic outbreak

While some reports show that the existing real-life medical resources allocations during epidemic outbreaks are myopic, some experts claim that medical resources allocations based on foresighted future allocations might enable a better balance of supply and demand. To investigate this claim, we develop a foresighted medical resources allocation model to help governments manage large-scale epidemic outbreaks. We formulate a demand forecasting model with a general demand forecasting function based on the last-period demands, extra demand caused by the last-period unfulfilled demand, and uncertain demand. In the foresighted allocation model, the government decides the current-period allocation based on the foresighted demand, which considers the last-period area demand and uncertain demand from the current period to the end of a planning horizon, using a stochastic dynamic program. We find that the optimal allocation is a function of the allocation capacity in each period. The optimal foresighted allocation is always higher than the optimal static (one-period) allocation and decreases with allocation capacity. When the allocation capacity is sufficiently large, the foresighted demand is close to the static demand. Besides, if the cost of oversupply is close to zero, the optimal allocations for both the foresighted allocation and one-period models are the allocation capacity. Our results provide useful managerial implications for a government contemplating medical resources allocation in response to an epidemic outbreak.

Pandemic portfolio choice

COVID-19 has taught us that a pandemic can significantly increase biometric risk and at the same time trigger crashes of the stock market. Taking these potential co-movements of financial and non-financial risks into account, we study the portfolio problem of an agent who is aware that a future pandemic can affect her health and personal finances. The corresponding stochastic dynamic optimization problem is complex: It is characterized by a system of Hamilton-Jacobi-Bellman equations which are coupled with optimality conditions that are only given implicitly. We prove that the agent’s value function and optimal policies are determined by the unique global solution to a system of non-linear ordinary differential equations. We show that the optimal portfolio strategy is significantly affected by the mere threat of a potential pandemic.

Crowdshipping: An open VRP variant with stochastic destinations

E-commerce continues to grow throughout the world due to people’s preference to stay at home rather than going to a brick-and-mortar retail store. COVID-19 has exacerbated this trend. Concurrently, crowd-shipping has been gaining in popularity due to both the increase in e-commerce and the current pressures due to COVID-19. We consider a setting where a crowd-shipping platform can fulfill heterogeneous delivery requests from a central depot with a fleet of professionally driven vehicles and a pool of capacitated occasional drivers. We divide delivery requests into sectors to represent different neighborhoods in a city. Occasional drivers have unknown destinations that can be anywhere inside the sectors. Route duration constraints are modeled to motivate participation and increase the probability of route-acceptance by keeping routes short. We assume that occasional drivers will choose routes that are better compensated and that the probability of route-acceptance is dependent on other routes being offered. We propose a two-stage stochastic model to formulate the problem. We use a branch-and-price algorithm capable of solving 50-customer instances, and develop a heuristic that can solve larger 100-customer instances quickly. An upper bound for the total number of occasional drivers is used to reduce the number of constraints in the master problem and reduce the complexity of the pricing problems. We show that occasional drivers with destinations far from the depot reduce the cost by over 30%, while occasional drivers with destinations that are near the depot reduce the cost by 20%. We show that route duration constraints and capacity constraints can restrict the occasional driver routes and both need to simultaneously increase in order to have cost reductions. This setting of crowd-shipping is a viable option for last-mile deliveries.

Improving the Effectiveness of e-Learning Processes through Dynamic Programming: A Survey

E-learning has been widely adopted as an important tool for distance education, especially in these days of pandemic Covid-19. However, several problems/challenges have been re-ported in different processes of e-learning that need to be ad-dressed for effective use of e-learning. These problems/challenges include development of student focused contents, giving learner partial control, addressing different learning styles, etc . Recently, several efforts have been made to solve e-learning process problems using dynamic programming techniques. Dynamic pro-gramming techniques divide a problem situation into several sub-problems and dynamically solves each sub-problem based on student needs. Thus it allows student focused customization at each step and provides adaptive e-learning to support students with different capabilities. The objective of this study is to review different e-learning problems and challenges and how those can be addressed using dynamic programming techniques. We conclude by highlighting the importance of different dynamic programming techniques for different processes of e-learning.

Optimal Time-Consistent Investment and Premium Control Strategies for Insurers with Constraint under the Heston Model

In this work, we study the optimal investment and premium control problem with the short-selling constraint under the mean-variance criterion. The claim process is assumed to follow the non-homogeneous compound Poisson process. The insurer invests the surplus in one risk-free asset and one risky asset described by the Heston model. Under these, we consider an optimization objective that maximizes the return (the expectation of terminal wealth) and minimizes the risk (the variance of terminal wealth). By constructing the extended Hamilton–Jacobi–Bellman (HJB) system with the dynamic programming method, the time-consistent strategies and the corresponding value function are obtained. Furthermore, we provide numerical examples to illustrate the effects of the model parameters on the optimal policies.

New class of optimal multiple stopping times problems

This paper is devoted to study a new discounted nonlinear optimal multiple stopping times problem with discounted factor β > 0 and infinite horizon. Under the right continuity of the underlying process, we show that the problem can be reduced to a sequence of ordinary optimal stopping problems. Next in the Markovian case, we characterise the value function of the problem in terms of β-excessive functions. Finally, in the special case of a diffusion process, we give explicit expressions for the value function of the problem as well as the optimal stopping strategy. As an explicit example in finance, we apply our theoretical results to manage a new generalised swing contract which gives its holder n rights to mark the price X of a stock, where the payment is only allowed at the final exercise time rather than at each exercise time as in the classical swing contact. Copyright © 2022 Inderscience Enterprises Ltd.

Optimal multi-stage group partition for efficient coronavirus screening.

The outbreak of COVID-19 has affected the economy worldwide due to entire countries being on lockdown. This has been highly challenging for governments facing constraints in terms of time and resources related to the availability of testing kits for the virus. This paper develops an optimal method for multiple-stage group partition for coronavirus screening using a dynamic programming approach. That is, in each stage, a group of people is divided into a certain number of subgroups, each will be tested as a whole. Only the subgroup(s) tested positive will be further divided into smaller subgroups in the next stage or individuals at the last stage. Our multiple-stage group partition scheme is able to minimize the total number of test kits and the number of stages. Our scheme can help solve the test kit shortage problem and save time. Finally, numerical examples with useful managerial insights for further investigation are presented. The results confirm the advantages of the multi-stage sampling method over the existing binary tree method.

OPTIMAL TALENT MANAGEMENT OF THE ACQUISITION WORKFORCE IN RESPONSE TO COVID-19: DYNAMIC PROGRAMMING APPROACH

The authors use a unique panel dataset of DoD civilian acquisition workers and a dynamic programming approach to simulate the impact of the pandemic on employee retention rates under a variety of recovery scenarios. While most world economies contracted in 2020, there is some consensus among economists of a relatively robust recovery in the near future, with average global economic growth projected to be about 5.5% in 2021 (International Monetary Fund, 2021). After calibrating the model parameters to the Defense Acquisition Workforce using a unique panel administrative personnel dataset that tracks the civilian DoD labor force over the span of 30 years, we simulate civilian-side labor market shocks that correspond to economic recoveries of varying speeds and forecast the retention behavior of the workforce. [...]the recovery of the rest of the world;additional federal, state, and local fiscal stimuli;as well as permanent changes in the economy, such as expanded work-from-home and reconfiguration of global supply chains, may impact the private-sector labor market for years to come.